Perspectives on label-free microscopy of heterogeneous and dynamic biological systems.

Significance: Advancements in label-free microscopy could provide real-time, non-invasive imaging with unique sources of contrast and automated standardized analysis to characterize heterogeneous and dynamic biological processes. These tools would overcome challenges with widely used methods that are destructive (e.g., histology, flow cytometry) or lack cellular resolution (e.g., plate-based assays, whole animal bioluminescence imaging). Aim: This perspective aims to (1) justify the need for label-free microscopy to track heterogeneous cellular functions over time and space within unperturbed systems and (2) recommend improvements regarding instrumentation, image analysis, and image interpretation to address these needs. Approach: Three key research areas (cancer research, autoimmune disease, and tissue and cell engineering) are considered to support the need for label-free microscopy to characterize heterogeneity and dynamics within biological systems. Based on the strengths (e.g., multiple sources of molecular contrast, non-invasive monitoring) and weaknesses (e.g., imaging depth, image interpretation) of several label-free microscopy modalities, improvements for future imaging systems are recommended. Conclusion: Improvements in instrumentation including strategies that increase resolution and imaging speed, standardization and centralization of image analysis tools, and robust data validation and interpretation will expand the applications of label-free microscopy to study heterogeneous and dynamic biological systems.

Dual-space disentangled-multimodal network (DDM-net) for glioma diagnosis and prognosis with incomplete pathology and genomic data.

Objective. Effective fusion of histology slides and molecular profiles from genomic data has shown great potential in the diagnosis and prognosis of gliomas. However, it remains challenging to explicitly utilize the consistent-complementary information among different modalities and create comprehensive representations of patients. Additionally, existing researches mainly focus on complete multi-modality data and usually fail to construct robust models for incomplete samples.Approach. In this paper, we propose adual-space disentangled-multimodal network (DDM-net)for glioma diagnosis and prognosis. DDM-net disentangles the latent features generated by two separate variational autoencoders (VAEs) into common and specific components through a dual-space disentangled approach, facilitating the construction of comprehensive representations of patients. More importantly, DDM-net imputes the unavailable modality in the latent feature space, making it robust to incomplete samples.Main results. We evaluated our approach on the TCGA-GBMLGG dataset for glioma grading and survival analysis tasks. Experimental results demonstrate that the proposed method achieves superior performance compared to state-of-the-art methods, with a competitive AUC of 0.952 and a C-index of 0.768.Significance. The proposed model may help the clinical understanding of gliomas and can serve as an effective fusion model with multimodal data. Additionally, it is capable of handling incomplete samples, making it less constrained by clinical limitations.

Integrating X-ray phase-contrast imaging and histology for comparative evaluation of breast tissue malignancies in virtual histology analysis.

Detecting breast tissue alterations is essential for cancer diagnosis. However, inherent bidimensionality limits histological procedures' effectiveness in identifying these changes. Our study applies a 3D virtual histology method based on X-ray phase-contrast microtomography (PhC µ CT), performed at a synchrotron facility, to investigate breast tissue samples including different types of lesions, namely intraductal papilloma, micropapillary intracystic carcinoma, and invasive lobular carcinoma. One-to-one comparisons of X-ray and histological images explore the clinical potential of 3D X-ray virtual histology. Results show that PhC µ CT technique provides high spatial resolution and soft tissue sensitivity, while being non-destructive, not requiring a dedicated sample processing and being compatible with conventional histology. PhC µ CT can enhance the visualization of morphological characteristics such as stromal tissue, fibrovascular core, terminal duct lobular unit, stromal/epithelium interface, basement membrane, and adipocytes. Despite not reaching the (sub) cellular level, the three-dimensionality of PhC µ CT images allows to depict in-depth alterations of the breast tissues, potentially revealing pathologically relevant details missed by a single histological section. Compared to serial sectioning, PhC µ CT allows the virtual investigation of the sample volume along any orientation, possibly guiding the pathologist in the choice of the most suitable cutting plane. Overall, PhC µ CT virtual histology holds great promise as a tool adding to conventional histology for improving efficiency, accessibility, and diagnostic accuracy of pathological evaluation.

Mitosis detection, fast and slow: Robust and efficient detection of mitotic figures.

Counting of mitotic figures is a fundamental step in grading and prognostication of several cancers. However, manual mitosis counting is tedious and time-consuming. In addition, variation in the appearance of mitotic figures causes a high degree of discordance among pathologists. With advances in deep learning models, several automatic mitosis detection algorithms have been proposed but they are sensitive to domain shift often seen in histology images. We propose a robust and efficient two-stage mitosis detection framework, which comprises mitosis candidate segmentation (Detecting Fast) and candidate refinement (Detecting Slow) stages. The proposed candidate segmentation model, termed EUNet, is fast and accurate due to its architectural design. EUNet can precisely segment candidates at a lower resolution to considerably speed up candidate detection. Candidates are then refined using a deeper classifier network, EfficientNet-B7, in the second stage. We make sure both stages are robust against domain shift by incorporating domain generalization methods. We demonstrate state-of-the-art performance and generalizability of the proposed model on the three largest publicly available mitosis datasets, winning the two mitosis domain generalization challenge contests (MIDOG21 and MIDOG22). Finally, we showcase the utility of the proposed algorithm by processing the TCGA breast cancer cohort (1,124 whole-slide images) to generate and release a repository of more than 620K potential mitotic figures (not exhaustively validated).

[Intraoperative stimulated Raman histology for personalized brain tumor surgery]. / Personalisierte Hirntumorchirurgie mittels intraoperativer stimulierter Raman-Histologie.

BACKGROUND: In brain tumor surgery a personalized surgical approach is crucial to achieve a maximum safe tumor resection. The extent of resection decisively depends on the histological diagnosis. Stimulated Raman histology (SRH), a fiber laser-based optical imaging method, offers the possibility for evaluation of an intraoperative diagnosis in a few minutes. OBJECTIVE: To provide an overview on the applications of SRH in neurosurgery and transference of the technique to other surgical disciplines. METHODS: Description of the technique and review of the current literature on SRH. RESULTS: The SRH technique was successfully used in multiple neuro-oncological tumor entities. Initial pilot projects showed the potential for analysis of extracranial tumors. CONCLUSION: The use of SRH provides a near real-time diagnosis with high diagnostic accuracy and provides further developmental potential to improve personalized tumor surgery.

Compartments in medulloblastoma with extensive nodularity are connected through differentiation along the granular precursor lineage.

Medulloblastomas with extensive nodularity are cerebellar tumors characterized by two distinct compartments and variable disease progression. The mechanisms governing the balance between proliferation and differentiation in MBEN remain poorly understood. Here, we employ a multi-modal single cell transcriptome analysis to dissect this process. In the internodular compartment, we identify proliferating cerebellar granular neuronal precursor-like malignant cells, along with stromal, vascular, and immune cells. In contrast, the nodular compartment comprises postmitotic, neuronally differentiated malignant cells. Both compartments are connected through an intermediate cell stage resembling actively migrating CGNPs. Notably, we also discover astrocytic-like malignant cells, found in proximity to migrating and differentiated cells at the transition zone between the two compartments. Our study sheds light on the spatial tissue organization and its link to the developmental trajectory, resulting in a more benign tumor phenotype. This integrative approach holds promise to explore intercompartmental interactions in other cancers with varying histology.

Protocol to study immunodynamics in the tumor microenvironment using a tyramide signal amplification-based immunofluorescent multiplex panel.

Immunodynamics in the tumor microenvironment can be precisely examined by using multiple antigen identification approaches. Here, we present a protocol for capturing expression levels of multiple target proteins in the same specimen at single-cell resolution using a tyramide signal amplification-based immunofluorescent multiplexing system. We describe steps for tumor tissue microarray preparation, multiplex immunohistochemistry staining, image acquisition, and quantification. This protocol can quantify immune cells in tissues from patients or experimental disease models at a protein level. For complete details on the use and execution of this protocol, please refer to Chung et al. (2023),1 Tang et al. (2022),2 and Tang et al. (2022).3.

A twin convolutional neural network with hybrid binary optimizer for multimodal breast cancer digital image classification.

There is a wide application of deep learning technique to unimodal medical image analysis with significant classification accuracy performance observed. However, real-world diagnosis of some chronic diseases such as breast cancer often require multimodal data streams with different modalities of visual and textual content. Mammography, magnetic resonance imaging (MRI) and image-guided breast biopsy represent a few of multimodal visual streams considered by physicians in isolating cases of breast cancer. Unfortunately, most studies applying deep learning techniques to solving classification problems in digital breast images have often narrowed their study to unimodal samples. This is understood considering the challenging nature of multimodal image abnormality classification where the fusion of high dimension heterogeneous features learned needs to be projected into a common representation space. This paper presents a novel deep learning approach combining a dual/twin convolutional neural network (TwinCNN) framework to address the challenge of breast cancer image classification from multi-modalities. First, modality-based feature learning was achieved by extracting both low and high levels features using the networks embedded with TwinCNN. Secondly, to address the notorious problem of high dimensionality associated with the extracted features, binary optimization method is adapted to effectively eliminate non-discriminant features in the search space. Furthermore, a novel method for feature fusion is applied to computationally leverage the ground-truth and predicted labels for each sample to enable multimodality classification. To evaluate the proposed method, digital mammography images and digital histopathology breast biopsy samples from benchmark datasets namely MIAS and BreakHis respectively. Experimental results obtained showed that the classification accuracy and area under the curve (AUC) for the single modalities yielded 0.755 and 0.861871 for histology, and 0.791 and 0.638 for mammography. Furthermore, the study investigated classification accuracy resulting from the fused feature method, and the result obtained showed that 0.977, 0.913, and 0.667 for histology, mammography, and multimodality respectively. The findings from the study confirmed that multimodal image classification based on combination of image features and predicted label improves performance. In addition, the contribution of the study shows that feature dimensionality reduction based on binary optimizer supports the elimination of non-discriminant features capable of bottle-necking the classifier.

Center-Focused Affinity Loss for Class Imbalance Histology Image Classification.

Early-stage cancer diagnosis potentially improves the chances of survival for many cancer patients worldwide. Manual examination of Whole Slide Images (WSIs) is a time-consuming task for analyzing tumor-microenvironment. To overcome this limitation, the conjunction of deep learning with computational pathology has been proposed to assist pathologists in efficiently prognosing the cancerous spread. Nevertheless, the existing deep learning methods are ill-equipped to handle fine-grained histopathology datasets. This is because these models are constrained via conventional softmax loss function, which cannot expose them to learn distinct representational embeddings of the similarly textured WSIs containing an imbalanced data distribution. To address this problem, we propose a novel center-focused affinity loss (CFAL) function that exhibits 1) constructing uniformly distributed class prototypes in the feature space, 2) penalizing difficult samples, 3) minimizing intra-class variations, and 4) placing greater emphasis on learning minority class features. We evaluated the performance of the proposed CFAL loss function on two publicly available breast and colon cancer datasets having varying levels of imbalanced classes. The proposed CFAL function shows better discrimination abilities as compared to the popular loss functions such as ArcFace, CosFace, and Focal loss. Moreover, it outperforms several SOTA methods for histology image classification across both datasets.

Neural Graph Refinement for Robust Recognition of Nuclei Communities in Histopathological Landscape.

Accurate classification of nuclei communities is an important step towards timely treating the cancer spread. Graph theory provides an elegant way to represent and analyze nuclei communities within the histopathological landscape in order to perform tissue phenotyping and tumor profiling tasks. Many researchers have worked on recognizing nuclei regions within the histology images in order to grade cancerous progression. However, due to the high structural similarities between nuclei communities, defining a model that can accurately differentiate between nuclei pathological patterns still needs to be solved. To surmount this challenge, we present a novel approach, dubbed neural graph refinement, that enhances the capabilities of existing models to perform nuclei recognition tasks by employing graph representational learning and broadcasting processes. Based on the physical interaction of the nuclei, we first construct a fully connected graph in which nodes represent nuclei and adjacent nodes are connected to each other via an undirected edge. For each edge and node pair, appearance and geometric features are computed and are then utilized for generating the neural graph embeddings. These embeddings are used for diffusing contextual information to the neighboring nodes, all along a path traversing the whole graph to infer global information over an entire nuclei network and predict pathologically meaningful communities. Through rigorous evaluation of the proposed scheme across four public datasets, we showcase that learning such communities through neural graph refinement produces better results that outperform state-of-the-art methods.

CoNIC Challenge: Pushing the frontiers of nuclear detection, segmentation, classification and counting.

Nuclear detection, segmentation and morphometric profiling are essential in helping us further understand the relationship between histology and patient outcome. To drive innovation in this area, we setup a community-wide challenge using the largest available dataset of its kind to assess nuclear segmentation and cellular composition. Our challenge, named CoNIC, stimulated the development of reproducible algorithms for cellular recognition with real-time result inspection on public leaderboards. We conducted an extensive post-challenge analysis based on the top-performing models using 1,658 whole-slide images of colon tissue. With around 700 million detected nuclei per model, associated features were used for dysplasia grading and survival analysis, where we demonstrated that the challenge's improvement over the previous state-of-the-art led to significant boosts in downstream performance. Our findings also suggest that eosinophils and neutrophils play an important role in the tumour microevironment. We release challenge models and WSI-level results to foster the development of further methods for biomarker discovery.

SynCLay: Interactive synthesis of histology images from bespoke cellular layouts.

Automated synthesis of histology images has several potential applications in computational pathology. However, no existing method can generate realistic tissue images with a bespoke cellular layout or user-defined histology parameters. In this work, we propose a novel framework called SynCLay (Synthesis from Cellular Layouts) that can construct realistic and high-quality histology images from user-defined cellular layouts along with annotated cellular boundaries. Tissue image generation based on bespoke cellular layouts through the proposed framework allows users to generate different histological patterns from arbitrary topological arrangement of different types of cells (e.g., neutrophils, lymphocytes, epithelial cells and others). SynCLay generated synthetic images can be helpful in studying the role of different types of cells present in the tumor microenvironment. Additionally, they can assist in balancing the distribution of cellular counts in tissue images for designing accurate cellular composition predictors by minimizing the effects of data imbalance. We train SynCLay in an adversarial manner and integrate a nuclear segmentation and classification model in its training to refine nuclear structures and generate nuclear masks in conjunction with synthetic images. During inference, we combine the model with another parametric model for generating colon images and associated cellular counts as annotations given the grade of differentiation and cellularities (cell densities) of different cells. We assess the generated images quantitatively using the Frechet Inception Distance and report on feedback from trained pathologists who assigned realism scores to a set of images generated by the framework. The average realism score across all pathologists for synthetic images was as high as that for the real images. Moreover, with the assistance from pathologists, we showcase the ability of the generated images to accurately differentiate between benign and malignant tumors, thus reinforcing their reliability. We demonstrate that the proposed framework can be used to add new cells to a tissue images and alter cellular positions. We also show that augmenting limited real data with the synthetic data generated by our framework can significantly boost prediction performance of the cellular composition prediction task. The implementation of the proposed SynCLay framework is available at https://github.com/Srijay/SynCLay-Framework.

DEPAS: De-novo Pathology Semantic Masks using a Generative Model.

The integration of artificial intelligence (AI) into digital pathology has the potential to automate and improve various tasks, such as image analysis and diagnostic decision-making. Yet, the inherent variability of tissues, together with the need for image labeling, lead to biased datasets that limit the generalizability of algorithms trained on them. One of the emerging solutions for this challenge is synthetic histological images. Debiasing real datasets require not only generating photorealistic images but also the ability to control the cellular features within them. A common approach is to use generative methods that perform image translation between semantic masks that reflect prior knowledge of the tissue and a histological image. However, unlike other image domains, the complex structure of the tissue prevents a simple creation of histology semantic masks that are required as input to the image translation model, while semantic masks extracted from real images reduce the process's scalability. In this work, we introduce a scalable generative model, coined as DEPAS (De-novo Pathology Semantic Masks), that captures tissue structure and generates high-resolution semantic masks with state-of-the-art quality. We demonstrate the ability of DEPAS to generate realistic semantic maps of tissue for three types of organs: skin, prostate, and lung. Moreover, we show that these masks can be processed using a generative image translation model to produce photorealistic histology images of two types of cancer with two different types of staining techniques. Finally, we harness DEPAS to generate multi-label semantic masks that capture different cell types distributions and use them to produce histological images with on-demand cellular features. Overall, our work provides a state-of-the-art solution for the challenging task of generating synthetic histological images while controlling their semantic information in a scalable way.

Primary Stability and Bone Contact Loading Evaluation of Suture and Screw based Coracoid Graft Fixation for Anterior Glenoid Bone Loss.

BACKGROUND: Reconstruction techniques for anterior glenoid bone loss have seen a trend from screws to suture-based fixations. However, comparative biomechanical data, including primary fixation and glenoid-graft contact pressure mapping, are limited. HYPOTHESIS: Suture-based bone block cerclage (BBC) and suspensory suture button (SB) techniques provide similar primary fixation and cyclic stability to double-screw fixation but with higher contact loading at the bony interface. STUDY DESIGN: Controlled laboratory study. METHODS: In total, 60 cadaveric scapulae were prepared to simulate anterior glenoid bone loss with coracoid autograft reconstruction. Graft fixation was performed with 3 different techniques: (1) an interconnected all-suture BBC, (2) 2 SB suspensions, and (3) 2 screws. Initial compression was analyzed during primary fixation. Cyclic peak loading with 50 N and 100 N over 250 cycles at 1 Hz was performed with a constant valley load of 25 N. Optical recording and pressure foils allowed for spatial bone block tracking and contact pressure mapping at the glenoid-graft interface. Load-to-failure testing was performed at a rate of 1.5 mm/s with ultimate load and stiffness measured. RESULTS: Initial graft compression was higher with screw fixation (141 ± 5 N) compared with suture-based fixations (P < .001), with BBC fixation providing significantly higher compression than SB fixation (116 ± 7 N vs. 91 ± 5 N; P < .001). Spatial bone block migration and ultimate failure load were similar between the BBC and screw groups. The SB group showed significantly increased bone block translation (3.1 ± 1.0 mm; P≤ .014) and rotation (2.5°± 1.4°; P≤ .025) and significantly lower ultimate failure load (180 ± 53 N) compared with the BBC (P = .046) and screw (P = .002) groups. Both suture-based fixations provided significantly increased graft-glenoid contact loading with higher pressure amplitudes (P≤ .032) and contact pressure after cyclic loading (+13%; SB: P = .007; BBC: P = .004) compared with screw fixation. CONCLUSION: Both SB and interconnected cerclage fixation improved dynamic contact loading compared with screw fixation in a biomechanical glenoid bone loss model. Cerclage fixation was biomechanically comparable with screw fixation but with a greater variability. SB fixation showed significantly lower primary fixation strength and greater bone block rotation and migration. CLINICAL RELEVANCE: Suture-based bone block fixations improved graft-glenoid contact loading, but the overall clinical consequence on healing remains unclear.

A Type IV Extension Supracondylar Fracture With Comminuted Fracture Pattern: A Case Report.

CASE: A three year, 11 month old girl sustained a right displaced extension supracondylar fracture (ESF) of the humerus with comminution of the lateral column after an indoor fall. At surgery, fracture reduction showed multidirectional instability. Adequate reduction was achieved by applying longitudinal traction of the arm with partial elbow flexion and forearm supination. One percutaneous medial pin, followed by one lateral cross pin, was used to immobilize the fracture. Normal posterolateral new periosteal bone formation was seen on radiograph on the lateral side. At 5-year follow-up, she had full range of asymptomatic and symmetrical elbow motion. CONCLUSION: This case report shows a displaced ESF with a comminuted lateral humeral column, which contributed to a lack of adequate lateral pin purchase on bone. A modified pin fixation technique first with a medial pin and followed by a lateral pin with both placed through the medial column was used for stable fracture fixation. In addition, this case showed that fracture comminution was a contributory factor to the rare multidirectional instability of the Gartland Type IV fracture.

Mueller matrix polarization parameters correlate with local recurrence in patients with stage III colorectal cancer.

The peri-tumoural stroma has been explored as a useful source of prognostic information in colorectal cancer. Using Mueller matrix (MM) polarized light microscopy for quantification of unstained histology slides, the current study assesses the prognostic potential of polarimetric characteristics of peri-tumoural collagenous stroma architecture in 38 human stage III colorectal cancer (CRC) patient samples. Specifically, Mueller matrix transformation and polar decomposition parameters were tested for association with 5-year patient local recurrence outcomes. The results show that some of these polarimetric parameters were significantly different (p value < 0.05) for the recurrence versus the no-recurrence patient cohorts (Mann-Whitney U test). MM parameters may thus be prognostically valuable towards improving clinical management/treatment stratification in CRC patients.

Multiplex Immunohistochemical Analysis of the Spatial Immune Cell Landscape of the Tumor Microenvironment.

The immune cell landscape of the tumor microenvironment potentially contains information for the discovery of prognostic and predictive biomarkers. Multiplex immunohistochemistry is a valuable tool to visualize and identify different types of immune cells in tumor tissues while retaining its spatial information. Here we provide detailed protocols to analyze lymphocyte, myeloid, and dendritic cell populations in tissue sections. Starting from cutting formalin-fixed paraffin-embedded sections, automatic multiplex staining procedures on an automated platform, scanning of the slides on a multispectral imaging microscope, to the analysis of images using an in-house-developed machine learning algorithm ImmuNet. These protocols can be applied to a variety of tumor specimens by simply switching tumor markers to analyze immune cells in different compartments of the sample (tumor versus invasive margin) and apply nearest-neighbor analysis. This analysis is not limited to tumor samples but can also be applied to other (non-)pathogenic tissues. Improvements to the equipment and workflow over the past few years have significantly shortened throughput times, which facilitates the future application of this procedure in the diagnostic setting.

Improved Labeling for "Transparent Mouse".

Techniques for clearing mouse tissues or whole bodies and making them transparent are useful for anatomic analysis, but labeling specific cell types has been difficult. Researchers have developed a method, called wildDISCO, to highlight these cells and improve visualization of structural details. The technique uses a solvent to remove cholesterol from cell membranes in the mouse's body and employs conventional antibodies to tag the cells.

Advances in Central Nervous System Tumor Classification.

Historically, the classification of tumors of the central nervous system (CNS) relies on the histologic appearance of cells under a microscope; however, the molecular era of medicine has resulted in new diagnostic paradigms anchored in the intrinsic biology of disease. The 2021 World Health Organization (WHO) reformulated the classification of CNS tumors to incorporate molecular parameters, in addition to histology, to define many tumor types. A contemporary classification system with integrated molecular features aims to provide an unbiased tool to define tumor subtype, the risk of tumor progression, and even the response to certain therapeutic agents. Meningiomas are heterogeneous tumors as depicted by the current 15 distinct variants defined by histology in the 2021 WHO classification, which also incorporated the first moelcular critiera for meningioma grading: homozygous loss of CDKN2A/B and TERT promoter mutation as criteria for a WHO grade 3 meningioma. The proper classification and clinical management of meningioma patients requires a multidisciplinary approach, which in addition to the information on microscopic (histology) and macroscopic (Simpson grade and imaging), should also include molecular alterations. In this chapter, we present the most up-to-date knowledge in CNS tumor classification, particularly in meningioma, in the molecular era and how it could affect their future classification and clinical management of patients with these diseases.

Short histological kaleidoscope - recent findings in histology. Part III.

The paper provides an overview of the current understanding of different cells' biology (e.g., keratinocytes, Paneth cells, myoepithelial cells, myofibroblasts, chondroclasts, monocytes, atrial cardiomyocytes), including their origin, structure, function, and role in disease pathogenesis, and of the latest findings in the medical literature concerning the brown adipose tissue and the juxtaoral organ of Chievitz.